FA 2386 - 14 - 1 - 0007 AOARD 134144 Final Report Robust Multimodal Cognitive Load Measurement II

This report summarizes the important research activities, study results and research accomplishments out of the Robust Multimodal Cognitive Load Measurement II (RMCLM) project in the past one-year period. The objective of this project includes research of the fundamental issues related to the use of multiple input modalities and their fusion to enable robust and automatic cognitive load measurement (CLM) in the real world. Firstly, we examined the multimodal behavioral model to include mouse interactivity streams and a modified sliding window implementation for CLM. The contemplation-style and hesitation-style features are used in the study. The effects of cognitive load on trust are also investigated under two different conditions: low load and high load by analysing behavioural signals such as mouse movement and physiological signals such as Galvanic skin response (GSR). Altogether, in the part one-year period, we had carried out CLM study with behavioral modalities such as mouse interactivity streams. The relations between cognitive load and trust were also analyzed with the use of mouse activities and GSR values. List of Publications a) Papers published in peer-reviewed journals 1. J. Zhou, J. Sun, F. Chen, Y. Wang, R. Taib, A. Khawaji, and Z. Li, “Measurable Decision Making with GSR and Pupillary Analysis for Intelligent User Interfaces”, ACM Transactions on Computer-Human Interaction (ToCHI), vol. 21, no. 6, article no. 33, 2015. b) Papers published in peer-reviewed conference proceedings: 2. J. Zhou, C. Bridon, F. Chen, A. Khawaji, and Y. Wang, “Be Informed and Be Involved: Effects of Uncertainty and Correlation on User’s Confidence in Decision Making”, SIGCHI Conference on Human Factors in Computing Systems (CHI2015), 2015. 3. A. Khawaji, J. Zhou, F. Chen, and N. Marcus, “Using Galvanic Skin Response (GSR) to Measure Trust and Cognitive Load in the Text-Chat Environment”, SIGCHI Conference on Human Factors in Computing Systems (CHI2015), 2015. 4. S. Arshad, Y. Wang, and F. Chen, “Interactive Mouse Stream as Real-Time Indicator of User’s Cognitive Load”, SIGCHI Conference on Human Factors in Computing Systems (CHI2015), 2015. 5. A. Khawaji, F. Chen, J. Zhou and N. Marcus, “Trust and Cognitive Load in the Text-Chat Environment: The Role of Mouse Movement”, Proceedings of OzCHI 2014. Distribution A: Approved for public release. Distribution is unlimited FA2386-14-1-0007 AOARD134144 Final Report Robust Multimodal Cognitive Load Measurement II